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Abstract — In this paper, we propose a general operating scheme 
which allows the utility company to jointly perform power 
procurement and demand response so as to maximize the social 
welfare. Our model takes into consideration the effect of the 
renewable energy and the multi-stage feature of the power pro- 
curement process. It also enables the utility company to provide 
quality-of-usage (QoU) guarantee to the power consumers, which 
ensures that the average power usage level meets the target value 
for each user. To maximize the social welfare, we develop a low- 
complexity algorithm called the welfare maximization algorithm 
(WMA), which performs joint power procurement and dynamic 
pricing. WMA is constructed based on a two-timescale Lyapunov 
optimization technique. We prove that WMA achieves a close- 
to-optimal utility and ensures that the QoU requirement is 
met with bounded deficit. WMA can be implemented in a 
distributed manner and is robust with respect to system dynamics 
uncertainty. 

I. Introduction 

Recent years have witnessed enormous efforts to modernize 
the power grid. Central to this challenging task are the inte- 
gration of renewable energy technologies lH] and the design 
of efficient user demand-response schemes f2l. In this paper, 
we propose a general operating scheme which allows the util- 
ity companies to optimally integrate the available renewable 
energy sources to maximize the social welfare, and to perform 
optimal demand response to ensure the users' quality-of-usage 
(QoU). 

Specifically, we consider a utility company, equipped with 
a renewable energy source, purchasing power from a two- 
stage electricity market to meet the users' demand. Every 
day, the utility company first purchases power from the day- 
ahead market for the next day, called the base-power, based 
on its forecast of the user demand and the available renewable 
energy. Then, if the base-power and the renewable power are 
enough to meet the user demand on the next day, no further 
action is needed; otherwise, the utility company purchases 
additional power from the real-time market to meet the power 
deficit, which incurs an extra cost. Besides performing power 
procurement, the utility company also adjusts its electricity 
selling prices to encourage the users to shift their deferrable 
loads to times when the prices in the market are low. At 
the same time, the utility company also has to ensure the 
users' quality-of-usage (QoU), i.e., the average user power 
consumption level exceeds a pre-agreed level. The objective of 
the utility company is to optimally perform power procurement 
and determine the electricity prices, so as to maximize the 
social welfare, i.e., aggregate user utility minus the cost for 
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supporting the user loads, subject to guaranteeing the users' 
QoU. 

There have been many previous works on integrating 
renewable energy into the power grid, designing optimal 
power procurement algorithms, and constructing demand re- 
sponse schemes. |3| proposes the concept of "risk-limiting- 
dispatching" as a way of incorporating renewable energy into 
the current power grid. Works |4| and [5] consider the problem 
of selling wind power in the current electricity market. ||6| 
designs contract schemes to best commercialize renewable 
energy for the electricity market. On the power procurement 
side, Q and lHJ consider the problem of optimally procuring 
power to maximize the social welfare for a single day under 
both independent and time-correlated demand conditions, us- 
ing stochastic subgradient methods. ||9j considers the problem 
of minimizing the expected power cost and formulates the 
problem as a Markov decision problem. ifTOl considers the 
problem of optimally utilizing the renewable energy for both 
cost minimization and revenue maximization. Ill] formulates 
the optimal power dispatch problem as a Markov decision 
problem. On the demand response side, [12| and lITSl for- 
mulate the problem of optimal demand response as convex 
programs and study the role of dynamic pricing. |14| uses 
a game theoretic approach to study the demand management 
problem. However, we note that most of the previous works (a) 
either consider the three problems separately, in which case the 
resulting algorithm may not be optimal for the entire system, 
or (b) do not consider the multi-stage nature of the power 
procurement process, which makes the solution less applicable 
to the power grid, or (c) only consider optimizing the system 
for a single period, in which case finding the optimal solution 
relies heavily on the convexity assumptions of the user utility 
functions. 

In this paper, we consider the problem of jointly optimizing 
power procurement and demand response to maximize the 
long term average social welfare. Our model fully accounts 
for the multi-stage nature of the electricity market and allows 
the user utility functions to be arbitrary increasing functions. 
Our algorithm design approach uses a two-stage Lyapunov 
optimization technique, which allows us to develop low- 
complexity optimal control algorithms. The main contributions 
of the paper are summarized as follows: 

• A general operating scheme for utility companies to 
jointly optimize power procurement and demand response 
to maximize the long term system welfares and ensure the 
users' quality-of-usage (QoU). 

• The optimal power procurement scheme for utility com- 
panies to maximize the gain of renewable energy sources. 
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Explicitly quantified the value of renewable energy to the 
system, and the effects of its mean and variance on the 
gain. 

• Characterization of the social welfare loss due to the 
constraint that all users must see the same price. 

• A low-complexity online control algorithm welfare max- 
imization algorithm (WMA), which fully accounts for 
the multi-stage nature of the electricity market, and is 
provably near-optimal for maximizing the social welfare 
while ensuring that the users' QoU are met with bounded 
deficit for all times. 

• An evaluation of our scheme using real-world renewable 
energy and market power prices data, and a demonstration 
that it effectively maximizes the social welfare. 

This paper is organized as follows. In Section|n] we describe 
our system model. In Section III we present our algorithm 



design approach and construct the welfare maximization al- 
gorithm (WMA) to jointly optimize power procurement and 



pricing. We then analyze our scheme in Section IV Numerical 
results are presented in Section |V] We conclude our paper in 
Section |Vl] 

II. System model 

We consider a system shown in Fig. [T] In the system, the 
utility company is supplying power to N users. Time is divided 
into days, and each day is further divided into T equal size 
slots that correspond, e.g., to one-hour 1 or 15-minute intervals. 
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(b) Frame structure 
Fig. L (a) The system modeL (b) The time structure. 

A. Power procurement model 

We assume that the utility company has a renewable energy 
source and denote the renewable power it obtains in slot t on 
day k by X{k,t). This renewable energy source can be solar 
panels or wind turbines owned by the utility company. We 
assume that X{k,t) = X{t) for some random variable X{t) 
and are i.i.d. every day. [^Here "=" represents "equal in distri- 
bution." We denote the probability density function (p.d.f.) of 
^{t) by fx{x,t), and compactly write the renewable energy 
as X{t) in the following. We denote the maximum capacity 

'This a.ssumption can be viewed as ignoring tlie potential time dependence 
of X{k, t) on k, e.g., tlie seasonal behavior change of wind power. However, 
our results can be extended to incorporate such dependence on k. 



of the renewable energy by Xmax, so that X{t) < Xmax with 
probability one. 

To serve the users, the utility company operates according 
to the following procedure. |^ 

Demand response: On day k — 1, the utility company 
first chooses the power price vector p{k) — {p{k,t),t ~ 

0. 1, T — 1) for day k from some feasible price set V, and 
announces the prices. We assume that the feasible price set V 
is a time-invariant and compact subset of M^, which includes 
the constraint < p(k, t) < pmax for all k, t. For instance, 
one example is V ^ [0, Pmax]"'"- 

Day-ahead planning: On day fc — 1, the utility company 
first forecasts the aggregate user demand L{k,t) based on its 
knowledge of the users' responses to the prices. We assume 
that the utility company can accurately forecast L{k, t), in- 
cluding its mean and the distribution of the randomness. |^ 

To efficiently utilize the renewable energy, the utility com- 
pany also forecasts the available renewable energy X{t) on 
day k based on fx{x,t). Then, it procures a base-power 
B{k,t) from the day-ahead market based on the estimated 
load L{k,t) and renewable energy X{t), at a price l3{k,t). 

Real-time purchasing: At time t on day k, if the procured 
power and the renewable energy are enough to meet the 
demand, i.e., B{k,t) + X{t) > L{k,t), no further action 
is needed. Otherwise, the utility company has to purchase 
additional power from the real-time market with a real-time 
price a{k,t) to meet the power deficit Y{k,t) = L{k,t) — 
B{k,t)-X{t). 

In the above steps, we assume that when purchasing 
power from the electricity market on day fc — 1, the util- 
ity company can observe the day-ahead price /3(fc, t) and 
the expected real-time a(fc, t). In practice, the day-ahead 
and real-time prices are time-varying. To model this as- 
pect, we denote /3(fc) = (/3(fc, 1), /3(fc, T)) and a(k) = 
(E[a(fc, 1)] , E[a(fc, T)] ), the market price vector pair. We 
then first assume that on each day, the pair {/3{k),a.{k)) 
is drawn i.i.d. from some finite market price set V'" = 
{(/3j , — 1, A/} according to some distribution which 

may not be known to the utility company. We will later extend 
our results to the case when the vector evolves according to 
some finite state Markovian process. Finally, we assume that 
both the day-ahead and real-time prices are upper bounded, 

1. e., < a{k,t) < amax and < /3{k,t) < Pmax, and denote 
Smax = nicix[amax: /^max]- 



B. User demand model 

We assume that each user derives a utility Un(k,t) = 
Un{L, t) at time t on day k by consuming a load of Lj^'"(t) < 
L < L'^^^, where is the minimum amount of load that 

user n must consume at time t, e.g., lighting at night. [^Each 

-In practice, there will be ancillary services in addition to the two stages 
considered here. Our model can be extended to incorporate these components. 

^Note that this is not very restrictive and can be done when the user utility 
functions are known. Even in the case when they are not known, the utility 
company can still predict with empirical data. 

^Our results readily extend to the case when U{-, ■) is also a function of 
k, in which case each day has different utility functions for each time. 
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Utility function Un{L,t) is only assumed to be an increasing 
and continuous function of L. 

Upon receiving the prices for day k, each user first plans 
an intended power consumption level L^^ik, t) for each time t 
(the choice of L'^^{k,t) will be specified later). However, the 
actual power consumption on day k may not be exactly equal 
to L^^{k,t). We model this effect by assuming that the actual 
consumption is equal to Ln{k,t) = L'^{k,t) + Wn{k,t) for 
some i.i.d. zero mean random variable Wn{k,t). We assume 
that Wn{k,t) = Wnit) for some random variable Wn(t) < 
w,™^ for all time. We denote the p.d.f. of w„(t) by fw^{t) 
and assume that w„(t) +L^(fc, t) > L'^'"it) for all times, and 
that Wn{t) is independent of the renewable energy X(t).|^ 

Each user determines his intended consumption by solving 
the following maximization problem, where the expectation is 
taken over w„(fc,t): 



L^(fc,t) = min arg max 



Here L?,- "^^'^ 



E C/„(L + u.„(fc,i),i) (1) 



^p{k,t){L + Wn{k,t)) 



is the maximum planned 



consumption level. That is, the user picks his load to minimize 
the expected utility minus cost. In the case when there are 
multiple choices, he picks the load with minimum magnitude. 
Therefore, the price vector uniquely determines the loads of 
the users. We also notice that, since Un{L,t) is a function 
of a single variable L, the maximization problem of user n 
can usually be efficiently solved with high accuracy, even if 
Un{L,t) is nonconcave. 

In practice, besides maximizing his utility via power con- 
sumption every time slot, each user also has a requirement 
on the average power usage level. To model this aspect, we 
assume that each user n has the following quality -of -usage 
(QoU) requirement: |^ 

K-lT-l 

liminf E E ^[Ln{k, t)] > ^ (2) 

^ °° fc=0 *=0 

Note that can also be viewed as measuring the willingness 
of user n to perform load shifting: a smaller repre- 
sents less stringent QoU requirement, and the user is more 
willing to defer his load. The constraint (j2]) can be viewed 
as an approximate version of the more stringent constraint 
Ym=o ^n{k,t) > TL^ for each frame k, which inevitably 
requires dynamic-programming as the solution technique |7|. 
As we will see later, (j2]i allows us to approximately solve the 
harder constraint with a much simpler online algorithm. 

Finally, to model the user heterogeneity, we assume that 
there exists a positive constant 7 > 1 such that: for any given 
price p{k, t) and for any user pair n and m, the intended 
consumptions are bounded by each other: 



Li{k,t) < ^Lt{k,t), il(fc,t) < jLi{k,t). 



(3) 



Note that in practice, user n's power consumption may well be correlated 
with the renewable energy level. This effect is captured by the utility function 
Un{L, t) in our formulation. 

*In our problem, this condition can be shown to be equivalent to requiring 
that the constraint holds with probability 1 when the limit is well defined. 



When 7 = 1, all the users are homogeneous; whereas if 
7 = 00, the users can be highly heterogeneous. Now by 
defining w{t) ^ En^nit) and Ld(fc,t) ^ E„i«(fc,0 to 
be the random and deterministic components of the load, the 
aggregate load can be expressed as: 



L{k,t) = L''{k,t) +w{k,t). 



(4) 



C. Utility company objective 

On day k, the cost for the utility company at time t is given 
by: 

Cost(fc, t) = (3{k, t)B{k, t) + a{k, t)Y{k, t), (5) 

where Y{k,t) = [L{k,t) - B{k,t) - X{t)]'^ denotes the 
power deficit at time t. The welfare obtained by the utility 
company-user system on day k is given by: 

T-l p AT -, 

Welfare(fc) = E ^Un{Ln{k,t),t) - Cost{k,t) . (6) 

t=0 '-,1=1 

Now for any feasibly power procurement and dynamic pricing 
policy n, its expected average social welfare is defined: 



Welfare 



n 



1 



liminf ^ V E 

k=0 



Welfare" (/c) 



(7) 



Here the expectation is taken over the random renewable 
energy X{t), the random market prices and Wn{t) for each 
user 

Below, we define Welfare^^ to be the optimal average wel- 
fare over all feasible policies that ensure the QoU constraint 
Q. The objective of the utility company is to find an optimal 
dynamic pricing and power procurement policy to maximize 
(|7]i subject to (j2]i. In the following, we refer to this problem 
as the Welfare Maximization Problem (WMP). To ensure the 
existence of at least one optimal feasible control policy, we 
assume that: 



> max L„ 

0<t<T-l 



Vn. (8) 



Note that the constraints in ([8]l simply say that the QoU level 
of each user is no less than the minimum requirement and is 
no more than the maximum consumption. We also assume that 
both the renewable energy X{t) and the load randomnesses 
Wn{t) are continuous random variables. 

III. Algorithm design 

In this section, we construct algorithms to solve the WMP 
problem. Our solution is based on the Lyapunov network op- 
timization technique |T5l and requires minimum information 
of the random dynamics in the system. To carry out our 
construction, we first present in Section |III-A| the optimal 
base-power procurement component of our algorithm, which 
maximizes the gain brought about to the utility company by 
the renewable energy. This result is not only interesting of its 
own, but also enables us to convert the Welfare(/c) function 
into a function only of the user prices p(fc, t), which facilitates 



the design of the dynamic response algorithm in Section III-B 



A. Optimal base-power procurement 

In this section, we study how the utility company can 
maximize the gain of the renewable energy. To do so, we first 
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define: 

Z{t)^X{t)-w{t), (9) 

as the "effective" random renewable energy, and use fz{z,t) 
and Fz{z,t) to denote the p.d.f. and c.d.f. of Z{t). We also 
use F^^{a,t) to denote the inverse c.d.f. function of Z{t), 
i.e., 

Fz\a,t) =mf{z : Fziz,t) a}. (10) 

We now state the following lemma, which gives the optimal 
power procurement scheme for the utility company to mini- 
mize its expected cost for meeting a load L(fc, t) at time t on 
day k. 

Lemma 1: The optimal base-power B*{k,t) is given by: 



0, 



if 



i{k,t) 



B*{k,t) 



L'^ik,t)-F^\ 



i(k,t) ■ 



t) 



P{k,t) 

else. 



When B*{k,t) = 0, we have: 

E[Cost*(fc,t)] = a(fc,i)E[(L^(fc,t) 
Else, if B*{k,t) > 0, we have: 
E[Cost*(A:,t)] = P{k,t)L'^{k,t) 



mr 



< 1 



(11) 



(12) 



'a{k, t) 



1, P(k,t) ,^ 



zfz{z,t)dz. (13) 



Moreover, E[Cost(fc,t)] is non-decreasing in L^{k,t). O 

Proof: See Appendix A. ■ 
Note that the second term on the right-hand-side (RHS) in 
( [T3] l can indeed be interpreted as the value of the renewable 
energy X{t) (denoted as VoR(t)) (without X{t), the minimum 
expected cost is f3(k,t)L'^{k,t)), i.e., 

r^z U(fc,f) 

VoR(t) = a(fc,t) / zfz{z,t)dz. (14) 



The following corollary follows from Lemma [T] and shows that 
the value of the renewable energy is increasing with its mean 
and non-increasing with its variance. 

Lemma 2: The value of renewable energy VoR(i) is in- 
creasing with the mean of X{t) and nondecreasing with the 
variance of X{t). 

Proof: See Appendix B. ■ 

B. The Lyapunov approach 

We now present the construction of the joint pricing-power 
procurement algorithm using the Lyapunov technique. The 
method works as follows: we first define a set of "deficit" 
queues for each of the constraints we want to guarantee, i.e.. 
Then, using the queueing dynamics (defined later), we ob- 
tain a drift inequality, which captures how the actions at every 
time affect the system welfare and the deficit queue sizes. 
After that, we construct our algorithm by finding our actions 
to minimize RHS of the drift inequality, which corresponds to 
finding the "descent" direction for the system state. By doing 
so at every time slot, we can guarantee that the average welfare 
is maximized. Also, although we start with the time average 
constraint (|2|, we can get as a by-product of our algorithm 
an explicit sample-path deficit queueing bound. This provides 



explicit delay guarantee for the deferred loads of the users, 
and is very useful in practice. 

To start, we define a load-deficit queue Qn{T), t — 0,1, ... 
for each user n as follows: Qn{0) — 0, and it evolves 
according to: 

Qnitk +< + !) = [Qnih +t)^ K{k, t)]+ + Lf:', (15) 

for all k and all t. Here t^. = kT is introduced for notation 
simplicity. The intuition behind the deficit queue is that, if we 
can guarantee Qn {t)/t — > 0, then (j2]) will be ensured. To see 
this, note from ([TSj that: 

Q«(r + 1) >Q„(r)-L„(r) + L^\ (16) 

Here t — + t. Taking expectations on both sides and 
summing it up from r = 0, ...r — 1, we get: 

r-l 

E[g„(r)] > E[Q„(0)] -^E[L„(t)] +rL^. (17) 

T = 

Dividing both sides by r, and taking the liminf as r — > oo, we 
see that (|2| follows. 

Now to make use of the deficit queues, we define a Lya- 
punov function V{t) = ^J2n=iQ^i'^) ™d denote Q{t) ^ 
(Q„(r),n = 1,...,N). We then define the following T-slot 
Lyapunov drift for every time tk — kT: 



A(tfe) =E 



Vitk) I Qitk) 



(18) 



Here the expectation is taken over the randomness of the re- 
newable energy and the potentially random power procurement 
and pricing action selections. 

We now proceed to construct our algorithm. We first have 



the following lemma regarding the drift defined in ( 18 i 
Lemma 3: For any value k = 0,1, we have: 
A{tk) < CT 



T-l 



-^E ^Q„(ife+<)[L„(fc,0-L^^] \Q{tk) 



(19) 



O 



HereC=iE„([ir]' + ra') 

Proof: See Appendix C. ■ 

To construct our algorithm, we choose a parameter e > 0, 
which represents how close we want our performance to be to 
the optimal value. We then define 77 = 1/e and substract from 



both sides of ( 19l the term 77E 



Welfare(A:) | Q{tk) 



to get: 



A(ife) -T^E 
< CT - 77E 

T-l 



Welfare(fc) | Q{tk) 
Welfare(fc) | Q{tk 



~Y.E J2Qn{tk+t)[Ln{k,t)-Lf;'] \Q{tk 

Now we plug in the definition of Welfare(fc) in (j6]l to get: 
Welfare(fc) | Q{tk 

T-l r N 



<CT-rjJ2^ J2UniLnik,t),t) ~ Cost{k,t) \ Q{tk) 



t=0 



71=1 



5 



T-1 

-EE 

t=0 



J2Qn(tk+t)[L„{k,t)~Ll'] \ Q{tk 



(20) 



T-l r N 



^{p{k)) = max 



EE 

t=0 



E 



1]Un{Ln{k,t),t) ~ 7]C0St{k,t) 



E- 



,(ife + t)i^(p(fc,t),i) I Q(ife) 



s.t. e V. 
Here p(fc) = {p{k,t),t 



^\pik,t))^ 



E 



^r]Un{p{k,t),t) - ?]Cost{k,t) 



+ 



QMLi{p{k,t),t) I Q(t,; 



L^{k,t) — niin arg max 



E 



Un{L + Wnik,t),t)(24) 



We can now construct our algorithm by choosing the actions 
that minimize the RHS of ( [20| . However, directly doing so will 
require solving the following dynamic programming problem: 



(21) 

0, ...,T — 1) is the price vector of 
day k, and V is the set of feasible prices defined in Section 

ED 

Therefore, we instead use an alternative method to approx- 
imately solve this dynamic programming problem, in which 
case our actions approximately minimize the RHS of ( [20| . 
The intuition behind our approach is that, since the queue 
sizes will only change by a finite amount in any slot, instead 
of using the exact deficit queue value in very time slot, we 
use the "less updated" information Q{tk)- By doing so, we 
can still approximately minimize the RHS of pO| . However, 
this approximation not only enables us to decouple the pricing 
decisions across time slots, which greatly reduces the com- 
putational complexity of our algorithm, but also allows us to 



-p{k,t){L + Wn{k,t)) 



• On Day k: The user consumes power, and the utility 
company performs real-time power balancing and deficit 
queue update: 

- User-Power-Consumption: Each user consumes an 
amount of power I/^(n, k) + Wn{k, t). 

— Real-Time-Power-Purchasing-and-Deficit- Update: 
The utility company procures power from the 
real-time market if necessary, and updates the deficit 
queues according to (15i. O 

Note that to implement the WMA algorithm, the utility 
company only has to use the instant value of market prices 
{f3{k),a{k)), and does not need to know their distribution. 
This greatly simplifies the implementation of the algorithm 
and makes it more robust. 

IV. Performance Analysis 

In this section, we analyze the performance of WMA. We 
first have the following theorem, which states that the optimal 
social welfare can be achieved by a stationary and randomized 
algorithm 11* of the following structure: on any day k ~ 1, 
the utility company observes the day-ahead and real-time 
price vector {(3{k), a.{k)), and chooses the price vector i.i.d. 



(/3,a) 



t = 0,...,T- l),m 



solve the pricing problem (( 23 i defined below) in a distributed 
manner using using the "consistency price" approach in 1 16] in 
a distributed manner without the users submitting their utility 
functions to the utility company. This makes our algorithm 
very suitable for practical implementation. 

Below, for the ease of presentation, we denote both £„(fc, t) 
and Un{L, t) as random functions of the price for convenience: 

Liik,t) = Liipik,t),t), Un{L,t) = U^{p{k,t),t). (22) 
Welfare Maximization Algoritlnm (WMA): 

• On Day fc — 1: the utility company performs pricing and 
and the users perform consumption planning for day k: 

- Utility-Company-Pricing: Based on its knowledge of 
the user demand, the utility company chooses the 
price p{k,t) for time tk + 1, so as to maximize: 

r N 



t = 



from some countable subset {{p. 
l,...,oo} of V according to some distribution {(i 
0, ...,T — 1),™ — l,...,oo}, both potentially depend on 
{(3{k),a.{k)). Then, under any prices, the utility company 



procures the base-power according to ( 1 1 



Theorem 1: The optimal system welfare Welfareg^ is given 
by the following optimization problem. 

Welfareav = sup {UtiHtygy - Costgv} (25) 



rT-l 



S.t. Utility ay = E 



Costav = E 



EE«- EE[c/«(pfr\^)] 



■ t=0 m=l 
T-l oo 



TLf < E 



2^ 2^ «mt 

t=0 m=l 

T-l oo 



(26) 



,(27) 



E E '^mt'"' E ^niPmnt^'t) 



(/3,a) ^ „ . 
Pnmt Pmt 



t=0 m=l 

(/3,a) 



Vn, TO, t, 



{pitV,t = 0, T - 1) e T', V (/3, a), TO, n (31) 

{pl^f\t^0,...,T-l)eV,yif3,a),m. (32) 

Here sup{} denotes the supremum. The outside expectations 
are taken over the day-ahead and real-time price vectors 
{f3{k), a.{k)). The inside expectations are taken over the user 
usage variations w„(i). O 

Proof: The proof is done by using Caratheodory's theo- 
rem and is similar to [ ISj . Hence, the details are omitted for 
brevity. ■ 



>0, 



E 



1, Vi,(/3,a), 



Vn,(28) 

(29) 
(30) 



s.t. p{k) e V. (23) 

- Utility-Company-Day-Ahead-Power-Procurement: 
Observe {(3{k), a.{k)), procure base-power B{k,t) 
according to (111 using knowledge of L{k,t) and 
Z{t) = X{t) -w{t). 

- User-Consumption-Planning: Choose the load 
L^{k,t) according to ([ij, i.e.. 



We see that the constraint (29 1 requires that the prices to 
all the users are the same for all time. From this result, we 
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get the following corollary, which characterizes the welfare 
loss due to the equal price constraint, which we call price-of- 
single-price, and denote as PoSP. 

Corollary 1: Denote Welfare gjj' the optimal social welfare 
when the utility company's prices can be set differently for 
each individual user, and WelfarCg,, the optimal value of of 
(|25]l without the constraint (|29|. Then, we have: 

Welfare^ll' = Welfai-e^^ > Welfare^^, (33) 

and the price-of-linear-pricing is given by: 

PoS P = Welfare^y - Welfare^^ . O (34) 

We can now present the performance results of the WMA 
algorithm in the following theorem. 

Theorem 2: (WMA with i.i.d. market prices) The average 
welfare achieved by WMA with 77 = 1/e satisfies: 

Welfare™* > Welfare^^ - CiT/r/. (35) 

Here Ci = f ^„ {[L'^^'^Y + [L^Y) is a constant independent 
of 77. Moreover, for all tk,k > and t, we have: 



get: 



N 

J2 Qn{tk + t)< S^^^Nj^rj + tY,LI 

71—1 n 



.0 



(36) 



We first see from Theorem [2] that the average welfare 
achieved by WMA is within 0(e) of the maximum. Thus, 
as we decrease the value of e (or equivalently, increase the 
value of 77), we can push the average welfare to be arbitrarily 
close to the maximum value. On the other hand, we also see 



from ( 36 1 that the deficit queues are deterministically upper 
bounded, implying that the users' QoU will be guaranteed 
with bounded deficit. Indeed, each user roughly has to defer 
an aggregate load of (5max7^?? + TL^ . This provides explicit 
delay guarantees to the users, and is very useful in practice. 
Note that the bound (|36]l is indeed a worst case upper bound. 
As we will see in Section [Vj the actual average deficit queue 
size can be much smaller 



Below we prove the queueing bound ( 36 1, the proof of ( 35 1 
will be presented in Appendix D. 



Proof: (Theorem |2| To prove (36 1, we show that it holds 
for every time tk,k > 1, using induction. First, we see that it 
holds for k — 1 because Qn(0) = and it increases by at 
most Y.n Ln every time slot. Hence ^„ Q„(ti) < r^„ Lf^ 
Now suppose (|36]l holds for fc = 1 , . . . , if , we prove that it 
also holds for k = K + 1. 

(I) First, if J2„Qn{tK) < ?7'5max^^7^ then (|36} holds for 
tK+i- This is because the maximum increase of X]n*5n(^) 
over any day is TJ2n ^n- 

(II) Now suppose J2n Qnitx) > ??<^max^7^- We will show 
that the term E [ - ?7Cost(is:, t) + ^„ Qn{tK + t)Li{k, t)] is 
an increasing function of L^{K,t) for all t on day K. Thus, 
WMA will encourage the users to consume more loads, which 
will then reduce the deficit queue sizes. To do so, we first see 
by Lemma [T] that: 

E [77Cost(/v , t)] < 77<5^ax (K, t) ■ (37) 



Now we fix 



with load L^, {K, t 



we can use 



If 
to 



Q,,{tK)Li{K,t) 



> 



> 



(38) 



This shows that the term E [ — ?7Cost(if , t) + 
J2nQnitK)Ln{K,t)] is increasing in L^^{K,t) throughout 
day K, which implies that the utility company will 
set the price to 0, and the users will always chooses 
Li{K,t) = L^;'^3\ Since L^' > L^v _,. ^^max^ jj^g ^jg^^jj 

queue sizes will not further increase on day K. This shows 
that Y.nQn{tK) > Y.nQn{tK+i), which by induction 
completes the proof of ( [36| . ■ 

Now we present the performance results of WMA under 
more general market price processes. 

Theorem 3: (WMA with Markov market prices) Suppose 
(/3(fc), <y.{k)) evolves according to some finite state irreducible 
and aperiodic Markov chain. Then, WMA with 77 — 1/e 
achieves: 

,WMA 



Welfare" 



> Welfai-e! 



N 

E 



Qn{tk + t) < <5maxiV72,7 + r^L^ 



.0 



(39) 



(40) 



Proof: (Theorem [3]l First we see that the queueing bound 
( |40l ) can be proven using the exact same argument as in the 
proof of Theorem [2] The performance bound ( 39 1 can be 
proven using a multiple-slot drift argument developed in 1 17|. 
The details are omitted for brevity. ■ 

V. Simulation 

In this section, we present simulation results for the WMA 
algorithm. In the simulation, we use T — 24 and consider 
there are 2 users. Note that here each user can be viewed as 
representing a class of consumers. We express the user load in 
unit of lOOMW. [] We assume that 3 if t ^ [9, 18], 

and i^'"(t) = 5 if t e [9, 18] for all users, and that Lj^^^ = 12. 
We assume that both users have the following utility function, 
which have units of 1000 dollars per 100 MW: 
8L te[9,18],L<5 
0.8L + 36 t e [9,18], L[5,6] 
4L + 16.8, t € [9,18], L e [6,12], 
amin(L,6), t ^ [9,18], L < 12. 
Here a is tunable parameter which allows us to study how 
the off-peak utility affects the social welfare. Since both users 
have the same utility function, we have 7 = 1. We note that the 
utility functions for time slots in [9, 18] are non-concave. We 
assume that the users have different QoU requirements, i.e., 
Lf = 4.5 and = 8. Thus, user 1 represents consumers that 
are more flexible in power consumption, and are more willing 
to shift their load. We also assume for simplicity that Wn (t) = 
for all t. Thus, each user consumes exactly the intended load. 
The feasible price set is chosen to be = [0,800], which 

^The number of users and the following per-user load parameters are chosen 
to match the annual average hourly power demand of PG&E for the whole 
year 2010, which is on the order of 4000 MW. 



UniL,t) = 



(41) 
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corresponds to 0<; to 8<; per kWh, and is consistent with the 
current electricity market prices. 

We also use the following real-world data for market prices 
and renewable energy. 

Hourly market prices: We first compute 12 hourly price 
data sets, for day-ahead and real-time market prices by averag- 
ing the data of PG&E in the year 2010 (One for each month. 
Data are from the CAISO daily report on the Federal Energy 
Regulatory Commission (FERC) website [18|). Then, on each 
day, we independently choose one price vector pair from the 
12 data set uniformly at random. We convert the prices to 
thousand dollars per lOOMW. 

Hourly renewable energy: We then compute the hourly 
renewable available energy using the recorded wind power 
of 100 wind turbines located near 40.427V, 124.391^ at the 
west coast in California in year 2006, which has an aggregate 
capacity of 300MW (data from the National Renewable En- 
ergy Laboratory (NREL) website |19 |). For each hour, we first 
average the values of the recorded wind power of that hour 
(sampled every 10 minutes) to obtain 365 samples (normalized 
to with unit lOOMW). We then use the samples as the state 
space and use the empirical distribution as its true distribution. 




diversity not only allows us to improve the social welfare 
by 2% (Fig. m to 9% (Fig. |3}, but also results in a smaller 
deficit (up to 41% saving in both cases). This is because 
allowing different prices enables the system to better adapt to 
the deficits, which ensures the QoU being met more quickly. 




Fig. 3. Average achieved welfare and average deficit queue size. The 
off-peak utihty coefficient a — 2. 

VI. Conclusion 

In this paper, we consider the problem of jointly optimizing 
power procurement and demand response to maximize social 
welfare. We develop a low-complexity algorithm called the 
welfare maximization algorithm (WMA), which is constructed 
based on a two-timescale Lyapunov optimization technique. 
We prove that WMA achieves a close-to-optimal utility and 
ensures that the users' average power usages meet their 
requirements with bounded deficit. We also perform simulation 
using real-world data and show that WMA effectively achieves 
near-optimal social welfare. 



Fig. 2. Average achieved welfare and average deficit queue size. The 
off-peak utiUty coefficient a — 3. 

We first simulate our algorithm with the off-peak utility 
coefficient a = 3. The simulation results are plotted in 
Fig. |2] and |3] We simulated WMA with both two pricing 
schemes: same-price-for-all and one-price-for-each (Note that 
in this case WMA will also converge to the corresponding 
optimal when different prices are allowed). From the results, 
we see that WMA efficiently improves the social welfare 
while effectively controlling the QoU. For instance, in Fig. [2] 
when f] = 20, the social welfare (under the same-price-for-all 
scheme) already reaches 19.6 thousand dollars per hour, which 
is near-optimal. Also, the corresponding deficit is roughly 37, 
which roughly corresponds to a 3 hour delay of the load, 
because the average hourly load is 5+8 — 13. We note that our 



deficit queueing bound (36i for 77 = 20 is 756. This suggests 



that the actual average deficit queue size of WMA can be much 
smaller than that in ( (36] l. 

From the simulation results, we can also see how the pricing 
scheme affects the social welfare. We see that this price 



Appendix A - Proof of Lemma[T] 

Here we prove Lemma [T| We first recall the Cost(fc,i) 
function: 

Cost(fc, t) = I3{k, t)B{k, t) + a(fc, t)Y{k, t), 

where the power deficit Y{t) is given by: 

Y{k,t) = [L{k,t) - X{k,t) - BikA)]^ 

= [L'^{k,t)- Z{k,t) - B{k,t)\^ . (42) 

We now present the proof For notation simplicity, we omit 
the k, t indexes when it is clear 

Proof: (Lemma [T]i First, it is clear that if a(k,t) = 
E[a(fc,t)] < I3{k,t), then B*{k,t) = 0. Now suppose 
a{k,t) > (3{k,t). Using ( [42] i and omitting the k,t indexes, 
we have: 

/oo 
[L'^-B- z] ^fz{z, t)dz + PB 
-QO 

[L"^ - B - z\fz{z,t)dz + PB. (43) 



— QO 

L'^-B 
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It can be verified that E[Cost(/c, i)] is convex in B. Thus, to 
minimize it, we take the derivate to get: 

dE|"C0St] r , H , rt si 

= f5 + a[-{L^-B)fz{L''-B,t)] 



dB 



-{L''-B)fz{L^-B,t) 
aFz{L'^ -B,t). 



Setting (44 1 to zero, we obtain: 



B* 



'L'-F,\^-,t) 



fz{z,t)dz 
(44) 

(45) 



This proves (fTTl). To see ( 13 1, suppose B* > and plug (45 1 
back into (|43|, we get: 

E[Cost(fc,t)] =/3S + a(L'^-S) / /^(z, i)<i2(46) 



-a I zfz{z,t)dz 



— oo 

1/fJ 



zfz{z,t)dz. 



(47) 



Here the second equality uses the fact that 

PI a. Finally, it can be seen 

from 



12 1 and 



13 I that E[Cost(/c,t)] is non-decreasing in 



(fc, tjT^This completes the proof 



Appendix B - Proof of Lemma[2] 

Here we prove Lemma [2] 

Proof: (Lemma |2]l Since w{t) is independent of X{t) and 
has zero mean, it suffices to show that VoR(i) is increasing 
with the mean of Z{t) and non-increasing with the variance 
of Z{t). To begin, we note that Z{t) can be written as a sum 
of a constant, which determines its mean, and a zero mean 
random variable, i.e., (we drop the time indexes below for 
convenience): 

Z = fi + aH. (48) 

Here H has unit variance, zero mean, and a pdf fnih)- cP' is 
the variance of Z. 

We first consider the case when B* > 0. Recall 

VoR(0 = a I zfz{z)dz. (49) 



Denote 9 ~ we can rewrite the above as: 



VoR(t) 



a / zfz{z)dz 



— oo 



{fi + ah)fH{h)dh 



oo 



fifH{h)dh + era 



hfH{h)dh 



auFni- 



hfH{h)dh. 



Now since Fz{0) = |, we see that Fh{^) 



= ^h\§)- Denote Oh = F^^{^), then we have: 

VoR(i) = + era / hfH{h)dh. (50) 



Now since E[_ff] = 0, we see that /^^ hfH{h)dh < 0. Hence 
VoR(t) is an increasing function of n and a non-increasing 
function of a. 

Now suppose B* = 0. In this case, we will show that 
E [Cost(/c, t)] is decreasing in n and increasing in a. To this 
end, we have: 

E[Cost] f {L'^- z)fz{z)dz 

J —oo 



— oo 



[L'^ - II- (jh)fH{h)dh 



fH{h)dh — aa 



hfnWdh. 



(51) 



We can now take derivative with respect to /i to get: 
aEfCostl ^L^-n^ 
dn a 
< 0. 

This implies that E[Cost] is decreasing with /i. We can also 
take a derivative with respect to cr to get: 

hfH{h)dh (52) 



9E Cost 



da 



> 0. 



In the last step, we have used the fact that E[iJ] = 0, thus 

/-OO ^''fH{h)dh < 0. We see that the function is increasing 
with cr. ■ 



Appendix C - Proof of Lemma[3] 

In this section, we prove Lemma |3] 

Proof: (Lemma [3]) Squaring both sides of the queueing 
dynamic equation (flSF, we have: 

[Qn{tk+t+l)]^ < [Qn{tk+t)]^ + [Ln(k,t)]^ + [Lfn^ 

-2Q„(fc,i) [Ln{k,t)~Lf^]. 

Multiplying both sides with ^, using the fact that L„(fc,t) < 
i™**, and summing over all n, we have: 

-Y,Qn(tk+t)[L„{k,t)-Lf:']. 

71 

Denote C ^ ^J2n ([-^^^ + [Ln?)- Summing over t = 
0, T — 1 and taking expectations on both sides conditioning 
on Q{tk), we get the following: 

A(ife) < CT (53) 
-^E ^Q„(tfe+t)[L„(fc,t)-Lf] \Q{h 



Thus This proves the lemma. 
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Appendix D - Proof of ([35]) of Theorem[2] 



In this section, we prove Theorem |2] To proceed, we 
first have the following lemma, which shows that WMA 



approximately minimizes (21 



Lemma 4: Let $* be the maximum value of <E>(p(fc)) and 
let p^(/c) — {p'^{k,t),t = 0, ...,T — 1) be the maximizers 
of $A(p(fc)) ^ J2^Z^ $^(p(A:,i)). Then $'^(pA(fc)) > $* - 
TCo, where Co - (T - 1) E„ + [L^/^. 

Proof: (Lemma [4]i From ( fTsj l, we have the following 
inequalities: 

0„(ife + < Qnitk) + tL^n, Qn{tk + t) > Qn{tk) " iL^'- 

Using these inequalities, we have: 

Qn(tfc+t)[i„(fc,i)-i^'] (54) 

< Qn(tfc)[in(fc,t) - L^] + [L^?)- 

Then, by comparing $(p(fc)) and "I>*(p(/c)), we see that for 
any p{k), 

t=0 n 



$(p(A:)) < 



< 



$^p(fc)) 4 



2 

r(r- 



maxi2 I r7-avi2\ 



Since this holds for aU p{k), it holds for the p{k) that maxi- 
mizes $(p(fc)). Thus, by defining ^ E„ + 
[L^^]^), we see that Lemma |4] follows. ■ 

We are now ready to prove Theorem |2] 

Proof: (Theorem |2| From Lemma [4] we see that WMA 
minimizes the RHS of ( |20] l to within TCq of the minimum. 
Thus we have the following inequality: 



(55) 



< CiT - V E 



Welfare™'^'^(fc) | Q{tk) 

N 

Y,U^{Ln{k,t)) -Cost^ {k,t) I Q{tk) 

71=1 



T-1 



t=0 

T-1 r 

- E E E + [^"(^' ^) - ] I 

Here Ci = C+Cq = ^{[L'^^f + [L^;;f), and the superscript 
A stands for any other alternative pricing policies, including 
the optimal randomized and stationary policy 11* defined in 
Theorem [T] Now plugging the policy H* into the RHS of (55 1 
and using the fact that: 



E 



Welfare"* (fc) 



= Welfare;v, E 



Ln{k, t) 



out a telescoping sum over fc = 0, 1, 



> L^:, Vti, (56) 



we immediately obtain: 



Welfare™*(fc) | Q{tk) 



<CiT- ryWelfaregy 



Note that in the above step, we have use the fact that the policy 
n* chooses actions purely as functions of the random system 
prices, and is independent of the deficit queue sizes. Now 
taking expectations on both sides over Q{tk) and carrying 



E 



L{tK) - L{to) 



-r^EE 



fe=0 



, ii" — 1, we get: 



Welfare™"* (fc) 



< KCiT - if?? Welfare;^. 

Rearranging the terms, using the fact that L{t) > for all t, 
and dividing both sides by rjK, we get: 

CiT E[L{to)] 



K-l 



-Ye 



k=0 



Welfare™"* (A:) 



> Welfare;^ - 



Kri 



Taking the liminf K 00 and using the fact that L{tf)) < 
00, we see that ([35| follows. ■ 
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